Efficient End-to-End Sentence-Level Lipreading with Temporal Convolutional Networks

Lipreading aims to recognize sentences being spoken by a talking face. In recent years, the lipreading method has achieved high level of accuracy on large datasets and made breakthrough progress. However, is still far from solved, existing methods tend have error rates wild data defects disappearing training gradient slow convergence. To overcome these problems, we proposed an efficient end-to-end sentence-level model, using encoder based 3D convolutional network, ResNet50, Temporal Convolutional Network (TCN), CTC objective function as decoder. More importantly, architecture incorporates TCN feature learner decode feature. It can partly eliminate RNN (LSTM, GRU) disappearance insufficient performance, this yields notable performance improvement well faster Experiments show that convergence speed are 50% than state-of-the-art method, improved 2.4% GRID dataset.